Memory Hierarchy Design for Stream Computing a Dissertation Submitted to the Department of Electrical Engineering and the Committee on Graduate Studies of Stanford University in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy

Several classes of applications with abundant fine-grain parallelism, such as media and signal processing, graphics, and scientific computing, have become increasingly dominant consumers of computing resources. Prior research has shown that stream processors provide an energy-efficient, programmable approach to achieving high performance for these applications. However, given the strong compute capabilities of these processors, efficient utilization of bandwidth, particularly when accessing offchip memory, is crucial to sustaining high performance. This thesis explores tradeoffs in, and techniques for, improving the efficiency of memory and bandwidth hierarchy utilization in stream processors. We first evaluate the appropriate granularity for expressing data-level parallelism entire records or individual words and show that record-granularity expression of parallelism leads to reduced intermediate state storage requirements and higher sustained bandwidths in modern memory systems. We also explore the effectiveness of softwareand hardwaremanaged memories, and identify the relative merits of each type of memory within the context of stream computing. Software-managed memories are shown to efficiently support coarse-grain and producer-consumer data reuse, while hardware-managed memories are shown to effectively capture fine-grain and irregular temporal reuse. We introduce three new techniques for improving the efficiency of off-chip memory bandwidth utilization. First, we propose a stream register file architecture that enables indexed, arbitrary access patterns, allowing a wider range of data reuse to